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Introduction 

^"U^The  overall  goal  of  this  project  is  to  understand  the  ability  of 
the  human  listener  to  extract  information  from  complex,  time-varying 
sounds  such  as  speech,  music  or  other  environmentally  important  signals. 
Specifically,  we  are  interested  in  the  listener's  ability  to  process 
modulations  of  frequency  and  amplitude  which  are  thought  to  carry  the 
information  in  such  signals.  To  that  end  we  have  devised  a  signal¬ 
processing  model  that  calculates  the  Envelope-Weighted  Average  of  the 
Instantaneous  Frequency  (EWAIF)  for  complex,  time-varying  signals. 
Previous  research  had  shown  that  the  listener's  ability  to  distinguish 
one  complex  sound  from  another  can  be  predicted  by  calculating^the  EWAIF 
value  for  each  sound. 

The  original  version  of  the  EW^IF^ model  processed  an  entire  signal, 
regardless  of  its  duration,  as  a  wfiole.  In  the  first  year  of  funding 
for  this  project,  we  revised  the  model  to  incorporate  a  temporal 
processing  window.  The  revised  model  acts  as  a  short-term  running 
averager:  A  brief  segment  of  the  signal  is  "captured"  within  the  window. 
The  EWAIF  calculatior^i's  performed  on  the  windowed  segment,  before  the 
window  is  move  in  time  to  capture  a  new  segment.  Our  original  version 
of  the  running-average  EWAIF  model  incorporated  two  parameters  to 
describe  the  windowing  process.  Both  the  width  of  the  window  and  the 
lag  (i.e.,  the  temporal  overlap)  were  specified.  Recent  efforts  to 
model  listener  performance  have  used  a  moving  average  "filter"  approach, 
thus  only  one  parameter  is  necessary. 

~  *^e  initiated  a  series  of  experiments  to  test  the  performance  of  the 
new  EWAIF  model.  Listeners  were  asked  to  discriminate  between  two 
frequency  modulated  tones.  For  one  tone,  the  GLIDE,  the  frequency  began 
"atT'a  givelTvaTcre  and'cKanged  linearly  over  time  to  a  final  value.  The 
extent  of  the  transition,  in  Hertz,  and  its  duration,  in  msec.,  were 
parameters  of  the  experiment.  For  the  target  signal  in  the  two-altern¬ 
ative,  forced-choice  task,  the  tone's  frequency  began  at  the  same 
frequency  and  moved  to  the  same  final  frequency  over  the  same  time 
period;  however,  the  trajectory  was  different.  The  target  signal 
followed  a  series  of  discrete  steps  in  moving  from  the  initial  to  final 
frequency.  For  a  small  number  of  discrete  steps  in  this  STEP  signal, 
the  listener  can  easily  distinguish  the  GLIDE  from  the  STEP.  As  the 
number  of  steps  is  increased,  and  thus,  the  extent  and  duration  of  each 
individual  step  is  reduced,  listeners  have  more  difficulty  in  making  the 
required  discrimination. 
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The  discrimination  of  STEP  vs.  GLIDE  signals  was  begun  in  spring 
semester  1988  at  the  University  of  Kansas.  In  August  1988,  the  PI  moved 
to  the  Speech  and  Hearing  Science  Division  at  the  Ohio  State  University. 
The  remaining  four  months  of  work  was  performed  under  a  research 
sub-contract  from  the  University  of  Kansas  to  the  Ohio  State  University 
Research  Foundation.  Data  collection  was  interrupted  for  the  months  of 
August  and  September  1988,  while  the  laboratory  equipment  was  being 
moved  from  Lawrence,  KS  to  Columbus  OH,  reassembled  and  tested.  Further 
delay  was  encountered  because  Ohio  State  is  on  a  quarter  system  and 
classes  did  not  begin  until  late  in  September.  We  were  unable  to 
recruit  a  new  group  of  listeners  until  the  students  returned  for  the 
fall  quarter. 

Data  collection  begun  in  the  fall  quarter,  continued  through  the 
end  of  the  period  covered  by  this  report  (Nov.  30,  1988)  into  the 
current  period  funded  directly  through  Ohio  State.  A  competing  renewal 
of  the  project  was  submitted  in  August  1988.  Work  on  the  project  has 
continued  since  Nov.  30,  funded  directly  through  Ohio  State. 

In  addition  to  the  experimental  portion  of  the  work,  the 
implementation  of  the  EWAIF  model  was  revised  and  extended.  Originally, 
we  implemented  the  model  within  the  ILS  signal  processing  package  on  the 
PDP  11/23  laboratory  computer  at  Kansas.  At  Ohio  state  we  have  moved 
upward  and  downward  from  the  11/23.  The  model  is  implemented  on  a  SUN 
workstation  system  using  the  MATLAB  package  and  also  on  a  PC  in  Turbo 
Pascal.  The  implementation  on  the  SUN  incorporates  the  front-end  of  the 
Patterson  Pulse  Ribbon  model  of  pitch  perception  which  simulates  the 
peripheral  filtering  of  the  auditory  periphery.  This  implementation  was 
accomplished  with  the  considerable  help  of  Ying  Yong  Qi,  then  a  doctoral 
student  in  Speech  and  Hearing  Science,  and  Ashok  Krishnumurthy, 

Assistant  Professor  of  Electrical  Engineering  at  Ohio  State.  Both 
continue  to  work  on  the  project  in  the  current  year. 

List  s£  research  objectives  and  current  status 

1.  continue  the  STEP  vs  GLIDE  discrimination  testing  at  other  signal 
frequencies. 

Discrimination  testing  was  begun  for  frequency  excursions  of  100- , 
200-  and  400  Hz,  centered  at  1  kHz  and  2  kHz,  with  durations  of  25-,  50- 
and  100  msec.  Work  was  completed  on  four  listeners  at  those  frequencies 
at  Kansas,  before  the  move  to  Ohio  state.  At  Ohio  State  three  new 
listeners  were  recruited  and  the  work  at  Kansas  was  partially  replicated 
to  ensure  that  the  laboratory  equipment  was  in  fact  not  damaged  by  the 
relocation.  The  listeners  at  Ohio  state  were  then  tested  at  500  Hz  and 
4  kHz.  Agreement  between  results  from  Kansas  and  those  from  Ohio  State 
is  extremely  good.  In  fact,  agreement  across  subjects  is  remarkable 
good. 


2 


Results  of  the  discrimination  testing  were  plotted  as  percent 
correct  discriminations  as  a  function  of  the  duration  of  an  individual 
step.  That  is,  if  the  STEP  signal  contained  four  steps  over  100  msec, 
the  P(C)  score  would  be  plotted  at  a  step  duration  of  25  msec.  P(C)  for 
discrimination  of  a  GLIDE  vs  50-msec  two-step  signal  would  be  plotted  at 
the  same  25  msec/step  point.  Psychometric  functions  were  grouped 
according  to  the  rate  of  change  of  the  frequency.  Thus,  we  have  8 
Hz/msec,  4  Hz/msec  and  2  Hz/msec  rates  for  the  conditions  tested. 

The  psychometric  functions,  grouped  as  described  above  (see  figures 
included  with  this  report)  reveal  several  very  interesting  effects. 
First,  extent  of  transition  in  Hertz,  or  duration  in  msec,  have  little 
effect  on  performance  when  the  psychometric  functions  are  grouped  by 
transition  rate.  Across  the  rates  tested,  performance  differs  little 
between  8  Hz/msec  and  4  Hz/msec.  At  2  Hz/msec  the  functions  are 
slightly  less  steep.  Performance  is  similar  at  500  Hz,  1  kHz  and  2  kHz 
for  the  listeners  tested  so  far.  At  4  kHz,  all  listeners  show  a  marked 
decrement  in  discrimination  performance.  The  task  is  much  more 
difficult  to  accomplish.  At  the  lower  center  frequencies,  the  75%  point 
estimate  of  a  temporal  window  indicates  a  value  of  7  to  10  msec. 
Estimates  are  more  difficult  to  determine  at  4  kHz,  because  of  the 
listeners'  inability  to  perform  the  task.  Our  preliminary  explanation 
for  the  break  down  in  performance  at  4  kHz  centers  on  the  assumption 
that  synchronization  of  nerve  fibers  must  be  necessary  for  the  listener 
to  track  the  frequency  transitions. 

Another  surprising  feature  of  these  results  is  the  lack  of  effect 
of  the  critical  bandwidth.  Except  at  4kHz,  the  transitions  sweep  across 
more  than  one  critical  bandwidth,  yet  the  listeners'  performance  is  not 
different  for  sweeps  within  one  critical  bandwidth  versus  sweeps  that 
traverse  two  or  more  critical  bandwidths. 

2.  Implement  a  multiple-band  version  of  the  EWAIF  model  to  handle  wide 
bandwidth  sounds. 

Patterson's  Pulse  Ribbon  Model  for  complex  sound  pitch  perception 
was  ported  to  a  SUN/3  workstation  system.  We  have  attempted  to  insert 
the  EWAIF  calculation  between  the  output  of  each  "critical  band"  filter 
and  its  hair  cell  emulator.  The  model  obtained  from  Patterson  is 
running  as  expected  for  non-dynamic  signals;  however,  the  interaction  of 
the  narrow  bandwidth  filters  with  the  swept-frequency  signals  used  in 
our  discrimination  experiments  has  led  to  some  unusual  results.  Figures 
included  with  this  report  indicate  the  problem.  The  instantaneous 
frequency  of  the  filtered  signal  from  each  band  shows  deviations  from 
the  expected  values.  It  appears  as  if  the  filter  begins  to  respond  at 
its  natural  frequency  when  the  signal  is  first  introduced.  As  the  swept 
tone  enters  the  pass  band,  the  tone  frequency  takes  over,  but  as  the 
tone  leaves  the  pass  band,  the  filter  response  moves  back  toward  the 
natural  frequency.  The  nature  of  these  deviations  depends  upon  both 
signal  and  filter  parameters.  Our  experimental ' results  described  above 
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reveal  none  of  these  interactions.  Different  implementations  of  the 
filter  bank  and  a  change  from  FFT  to  FIR  filter  implementations  of  the 
Hilbert  transform  have  not  altered  this  effect.  These  preliminary  model 
results  further  support  the  notion  that  our  swept  tone  signals  are 
processed  without  the  influence  of  the  critical  band. 

3.  Complete  single-step  vs  glide  discrimination  experiments. 


The  series  of  single-step  vs  glide  discrimination  experiments  was 
completed  in  summer  1988  by  R.  A.  Gerren,  In  addition,  listeners  were 
tested  on  discrimination  of  Huffman  sequence  signals  like  those  used  by 
Patterson  and  Green  in  their  temporal  acuity  study  in  1970.  A 
manuscript  is  being  written  by  Gerren  and  Feth  for  submission  in  1989. 


Earti.gipat.ing  professionals 
Lawrence  L.  Feth,  Ph.D. 
Lisa  J.  Stover,  M.A. 
Richard  A.  Gerren,  Ph.D. 
Mary  E.  Neill,  B.A. 

Ying  Yong  Qi,  M.S. 

Ashok  Krishnumurthy,  Ph.D. 


Principal  Investigator 
Grad.  Research  Asst. 
Postdoctoral  fellow 
Grad .  Research  Asst . 
Grad .  Research  Asst . 
Co-Investigator 


(to  8/1/88) 
(to  8/1/88) 
(fr  8/1/88) 
(fr  8/1/88) 
(fr  12/1/88) 


Note  R.  A.  Gerren  was  funded  by  an  NIH  institutional  postdoctoral 
fellowship  awarded  through  the  University  of  Kansas.  Y.  Y.  Qi  was 
funded  by  an  Ohio  State  pre-doctoral  fellowship  for  the  period  covered 
by  this  report.  As  of  March  1,  1988  he  will  be  funded  by  a  graduate 
school  postdoctoral  fellowship  to  continue  until  Nov  30,  1989.  From  Dec 
1,  1989  on,  he  will  be  funded  through  this  project. 

Eufeljg.atiQ.ns  and  Presentations 


Complex  sound  discrimination:  Predictions  of  the  EWAIF  model.  L.L. 
Feth,  L.J.  Stover  and  R. A.  Gerren.  [Abstract:  J.  Acoust.  Soc.  Amer. 
83,  p.  S35  (1988)]  Presentation  to  the  Acoustical  Society  meeting  May 
1988,  Seattle  WA. 


Demodulation  Processes  in  Auditory  Perception,  L.L.  Feth, 
presentation  to  the  Bioacoustics  Laboratory  Group  at  Wright-Patterson 
AFB  Dec  4,  1988,  and  to  the  Central  Ohio  Chapter  of  the  Acoustical 
Society  of  America,  Feb  4,  1989. 

Envelope-Weighted  Average  of  Instantaneous  Frequency  Model  for 
Auditory  Processing  of  Complex  Sounds,  L.L.  Feth,  L.J.  stover,  R.A. 
Gerren,  and  M.  E.  Neill.  (Presented  at  the  twelfth  mid-winter  meeting 
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of  the  Association  for  Research  in  Otolaryngology,  Feb  1989.) 

Patents  and  Inventions 

No  patentable  inventions  have  resulted  from  this  research. 
Statements 

In  the  previous  annual  report  it  was  stated  that  a  manuscript  for 
publication  was  anticipated  by  spring/summer  of  1988.  Obviously,  that 
manuscript  has  not  been  completed.  The  main  reason  for  the  lack  of  a 
finished  manuscript  was  the  Pi's  decision  to  move  from  the  University  of 
Kansas  to  Ohio  state.  In  addition  to  the  actual  down  time  resulting 
from  the  move,  the  PI  had  to  negotiate  the  release  of  grant  related 
equipment  from  Kansas,  write  a  sub-contract  agreement  for  the 
continuation  of  this  work  at  Ohio  state,  and  write  a  three-year  renewal 
proposal  to  support  the  work  beyond  Nov  30,  1988.  From  March  11  through 
Sept  7,  1989,  the  PI  will  be  visiting  the  laboratory  of  Roy  Patterson  at 
the  Applied  Psychology  Unit  of  the  Medical  Research  Council  in 
Cambridge,  England.  The  purpose  of  the  visit  is  to  work  on  the 
combination  of  EWAIF  model  dynamic  features  with  the  multi-channel 
features  of  Patterson's  pulse  ribbon  model  of  pitch  perception.  In 
addition,  it  should  be  possible  to  complete  one  or  more  manuscripts 
during  the  visit. 

The  move  to  Ohio  State  was  made  because  of  that  university's 
renewed  commitment  to  support  research  in  speech  and  hearing  science. 
Over  the  past  year,  Ohio  state  has  committed  funds  to  house  the  speech 
and  hearing  division  in  remodeled  space  in  Pressey  Hall  on  the  Ohio 
State  campus.  Two  full  professors  (the  PI  and  one  other)  were  hired  as 
additions  to  the  faculty.  Further  additions  for  next  year  include  a 
speech  scientist  with  a  Ph.D.  in  physics  to  work  on  articulatory  models 
of  speech  production.  The  university  has  also  expended  approximately 
$500,000  to  the  purchase  of  research  equipment  for  speech  and  hearing 
science.  The  commitment  to  support  research  in  this  area  is  apparent. 
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ENVELOPE-WEIGHTED  AVERAGE  OF  INSTANTANEOUS  FREQUENCY  MODEL 
FOR  AUDITORY  PROCESSING  OF  COMPLEX  SOUNDS.  *  L.L.  Feth1.  L.J.  Stover2, 
R.A.  Gerren2,  M.E.  hleiH^  ^Speech  and  Hearing  Science,  Ohio  State  University, 
Columbus,  OH  43210,  2KASPL,  University  of  Kansas,  Lawrence,  KS  66045. 

The  assumption  underlying  this  work  is  that  the  human  auditory  system  functions 
as  an  auditory  signal  processor.  We  consider  the  complex  stream  of  sound  to  be  a  carrier 
modulated  in  amplitude  and  frequency.  To  extract  the  information  borne  by  the  carrier,  the 
listener  must  be  able  to  demodulate  the  sound  stream.  Much  of  the  past  work  in 
psychoacoustics,  and  some  of  the  present  day  effort,  is  based  in  what  we  call  "spectrum 
picture  processing".  That  is,  the  complex  sound  is  Fourier  analyzed  into  an 
amplitude-by-firequency  picture  in  the  experimenter's  conception  of  the  stimulus.  Then, 
perception  is  modeled  as  if  the  spectral  picture,  not  the  complex,  time-varying  sound 
pressure  wave,  were  the  stimulus  presented  to  the  listener. 

We  have  applied  an  earlier  version  of  the  EWAIF  model  to  simultaneously 
amplitude-  and  frequency-modulated  tones  to  predict  the  pitch  that  listeners  perceive  in  such 
sounds.  The  model  even  predicts  some  confounding  of  listener  performance  for  the  signals 
used  in  the  early  profile  analysis  work.  Our  current  implementation  of  the  model  tracks 
transitions  in  frequency  and  amplitude  for  narrow  bandwidth  signals.  To  test  the  model,  we 
predict  listener  performance  in  distinguishing  smooth  linear  transitions  from  multiple-step 
transitions  over  the  same  trajectory.  For  transitions  centered  from  250  Hz  to  4000  Hz,  at 
rates  of  2-,  4-,  and  8-Hz/sec,  we  find  that  listener  performance  can  be  characterized  as  a 
demodulation  process.  Limitations  on  the  performance  of  our  listeners  will  be  compared 
with  the  predictions  of  the  EWAIF  model. 

(Work  supported  by  a  grant  from  AFOSR) 
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PERSONAL:  Date  of  Birth:  September  25,  1957 
Place  of  Birth:  Madras,  India 

AREAS  OF  INTEREST: 

Speech  Analysis,  Synthesis  and  Recognition;  Digital  Signal  Processing;  Neural 
network  applications  in  speech  recognition. 


EDUCATION: 

Ph.D.  in  Electrical  Engineering,  University  of  Florida,  Dec.  1983. 

M.E.  in  Electrical  Engineering,  University  of  Florida,  May  1981. 

B.Tech.  in  Electrical  Engineering,  Indian  Institute  of  Technology,  Madras,  In¬ 
dia,  May  1979. 


EXPERIENCE: 

Assistant  Professor,  Department  of  Electrical  Engineering,  The  Ohio  State 
University,  Columbus,  Ohio,  September  1986  -  present. 

Assistant  Professor,  Computer  Science  and  Engineering  Department,  Auburn 
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Visiting  Assistant  Professor,  Department  of  Electrical  Engineering,  University 
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Department,  Auburn  University,  1985-86. 
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SOCIETY  MEMBERSHIPS: 


Institute  of  Electrical  and  Electronics  Engineers 

-  Computer  Society 

-  Acoustics,  Speech  and  Signal  Processing  Society 

-  Communications  Society 
Association  for  Computing  Machinery 
Acoustical  Society  of  America 

Eta  Kappa  Nu 


PROFESSIONAL  ACTIVITIES: 

Reviewer,  IEEE  Transactions  on  Acoustics,  Speech  and  Signal  Processing. 
Member,  IEEE  Computer  Society  Area  Activities  Board,  1985. 
Vice-Chairman,  Central  Ohio  Chapter  of  the  Acoustical  Society  of  America. 


CONSULTING: 

Summer  1986,  National  Semiconductor  Corporation,  Santa  Clara,  CA. 

PUBLICATIONS: 

Journal  Articles 

“A  new  competitive  learning  algorithm  for  vector  quantization  using  neural 
networks,”  submitted  to  Neural  Networks ,  November  1988,  (with  S.  C.  Ahalt, 
P.  Chen,  and  D.  E.  Melton). 

“A  modified  frequency-weighted  Itakura  spectral  distortion  measure,”  ac¬ 
cepted  for  publication  in  IEEE  Transcations  on  Acoustic ,  Speech  and  Signal 
Processing ,  (with  J.  Li). 

“Two  channel  speech  analysis,”  IEEE  Transactions  on  Acoustic,  Speech  and 
Signal  Processing ,  vol.  ASSP-34,  no.  4,  pp.  730-743,  August  1986,  (with 
D.  G.  Childers). 

“A  critical  review  of  electroglottography,”  CRC  Critical  Reviews  in  Biomedical 
Engineering ,  vol.  12,  no.  2,  pp.  131-161,  1985,  (with  D.  G.  Childers). 

“Voiced  Speech  Model  including  Source-Tract  Interaction,”  submitted  to  Speech 
Communication,  September  1988,  (with  J.  Li). 
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Conference  Papers 


“A  new  neural  network  learning  algorithm  for  vector  quantization,”  Micro¬ 
electronics  and  Photonics  in  Communication  Workshop,  June  1989,  (with 
S.  C.  Ahalt,  P.  Chen,  and  D.  E.  Melton). 

“Performance  of  synthetic  neural  network  classification  of  noisy  radar  signals,” 
to  appear  in  the  Proceedings  of  the  IEEE  Conference  on  Neural  Information 
Processing  Systems,  1988,  (with  S.C.  Ahalt,  F.D.  Garber,  and  I.  Jouny). 

“Formant  Estimation  and  Vowel  Recognition  in  Noise,”  Seventh  Annual  Con¬ 
ference,  American  Voice  Input/Output  Society,  Oct.  1988,  (with  J.  Li). 

“Formant  Estimation  from  Noisy  Voiced  Speech,”  115th  Meeting  of  the  Acous¬ 
tical  Society  of  America,  Journal  of  the  Acoustical  Society  of  America,  Suppl. 

I,  83,  Si  (1988),  (with  J.  Li  and  R.  L.  Moses). 

“Voiced  Speech  Model  including  Source-Tract  Interaction,”  115th  Meeting 
of  the  Acoustical  Society  of  America,  Journal  of  the  Acoustical  Society  of 
America,  Suppl.  1,  83,  S2  (1988),  (with  J.  Li). 

“Noisy  speech  analysis  using  the  speech  and  electroglottograph  signals,”  in 
Proceedings  of  the  18th  Southeastern  Symposium  on  System  Theory,  Knoxville, 
TN,  April  1986,  (with  F.  E.  Tokay). 

“Two  channel  (speech  and  EGG)  analysis  for  formant  tracking  and  glottal 
inverse  filtering,”  in  Proceedings  of  the  International  Conference  on  Acoustics, 
Speech  and  Signal  Processing,  San  Diego,  CA,  March  1984. 

“Glottal  sensing  for  speech  analysis  and  synthesis,”  in  Proceedings  of  the  In¬ 
ternational  Conference  on  Acoustics,  Speech  and  Signal  Processing,  Boston, 
MA,  April  1983,  (with  J.  J.  Yea,  J.  M.  Naik  and  D.  G.  Childers). 

“Vocal  fold  vibratory  patterns:  Comparison  of  film  and  inverse  filtering,”  in 
Proceedings  of  the  International  Conference  on  Acoustics,  Speech  and  Signal 
Processing,  Atlanta,  GA,  March  1981,  (with  D.  G.  Childers). 

“Laryngeal  function:  Assessment  and  role  in  speech  analysis  and  synthesis,”  in 
Proceedings  of  the  10th  International  Congress  of  Phonetic  Sciences,  Utrecht, 
The  Netherlands,  1983,  (with  D.  G.  Childers,  J.  J.  Yea  and  G.  P.  Moore). 

“Spectral  Analysis:  AR,  MA  or  ARMA,”  in  Proceedings  of  the  1st  ASSP 
Workshop  on  Spectral  Estimation,  August  1981,  (with  D.  G.  Childers  and 

J.  J.  Yea). 
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Book  Chapters 

“Vocal  source  and  tract  models  based  on  speech  signal  analysis,”  in  Mathemat¬ 
ics  and  Computers  in  Biomedical  Applications ,  J.  Eisenfeld  and  C.  DeLisis, 
Eds.,  pp  335-349,  Elsevier  Science  Publishers,  1985,  (with  D.  G.  Childers, 
E.  L.  Bocchieri  and  J.  M.  Na£). 

“Electroglottography,  speech  and  ultra-high  speed  cinematography,”  in  Vocal 
Fold  Physiology ,  I.  R.  Titze  and  R.  C.  Scherer,  Eds.,  pp  202-220,  The  Den¬ 
ver  Center  for  the  Performing  Arts,  1983,  (with  D.  G.  Childers,  J.  M.  Naik, 
J.  N.  Larar  and  G.  P.  Moore). 
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